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PROPOSED AMENDMENT 

Applicants thank the Examiner for suggestions regarding moving the application 
forward. Applicants propose the following amendments to place the application in 
condition for allowance. The Examiner indicated that only independent claims 1 and 2 
would be examined. Before canceling the other independent claims (15, 23, 25, 37, 40, 
and 41), Applicants require a written Restriction. 

Please note that Applicants have amended claims 1 and 2 by removing 
"comprising tenure." Applicants believe the claims are patentable without such 
language. 

Finally, Applicants propose adding claim 51, a product by process claim that 
mimics the language of claim 1. 
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Claims 



1 . (Currently Amended) A method of constructing a model operable to 
generate one or more job performance criteria predictions based on input pre-hire 
information, the method comprising: 

electronically collecting pre-hire information from a plurality of applicants 
wherein at least some of the pre-hire information is collected from at least one of the 
applicants who responds directly on an electronic device to provide pre-hire applicant 
responses to questions; 

collecting post-hire information for the applicants based on job performance of the 
applicants after hire; 

via information-theoretic feature selection, choosing questions from the pre-hire 
information as features for which respective pre-hire applicant responses serve as inputs 
to the model, wherein the information-theoretic feature selection comprises identifying at 
least one higher-order interaction comprising a set of a plurality of questions having 
higher predictive power than a sum of predictive powers of individual questions in the 
set , wherein the higher-order interaction exhibits a synergy between the set of the 
plurality of questions having higher predictive power ; and 

from the pre-hire information and the post-hire information, training an artificial 
intelligence-based predictive model in a computer-readable medium with observed pre- 
hire applicant responses for the chosen features, wherein the artificial intelligence-based 
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predictive model is operable to generate one or more job performance criteria predictions 
compri s ing t e nur e based at least on input pre-hire information from new applicants 
corresponding to the chosen features, whereby the one or more job performance criteria 
predictions are usable as a basis for a hiring recommendation or other employee selection 
information^ 

deploying the model, wherein deploying comprises converting the model into 
command code and providing an operational applicant processing system; and 

conducting performance tuning for the model, wherein performance tuning 
comprises continuing data collection, monitoring sample size as incoming data 
accumulates., and repeating feature selection . 

2. (Currently Amended) A computer-readable medium comprising computer- 
executable instructions for performing a method of constructing a model operable to 
generate one or more job performance criteria predictions based on input pre-hire 
information, the method comprising: 

electronically collecting pre-hire information from a plurality of applicants 
wherein at least some of the pre-hire information is collected from at least one of the 
applicants who responds directly on an electronic device to provide pre-hire applicant 
responses to questions; 

collecting post-hire information for the applicants based on job performance of the 
applicants after hire; 
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via information-theoretic feature selection, choosing questions from the pre-hire 
information as features for which respective pre-hire applicant responses serve as inputs 
to the model, wherein the information-theoretic feature selection comprises identifying at 
least one higher-order interaction comprising a set of a plurality of questions having 
higher predictive power than a sum of predictive powers of individual questions in the 
set , wherein the higher-order interaction exhibits a synergy between the set of the 
plurality of questions having higher predictive power : and 

from the pre-hire information and the post-hire information, training an artificial 
intelligence-based predictive model with observed pre-hire applicant responses for the 
chosen features, wherein the artificial intelligence-based predictive model is operable to 
generate one or more job performance criteria predictions compri s ing tenure based at 
least on input pre-hire information from new applicants corresponding to the chosen 



deploying the model, wherein deploying comprises converting the model into 
command code and providing an operational applicant processing system: and 

conducting performance tuning for the model wherein performance tuning 
comprises continuing data collection, monitoring sample size as incoming data 
accumulates, and repeating feature selection . 



features^ 



3. 



(Canceled) 
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4. (Canceled) 

5. (Original) The method of claim 1 further comprising; 

limiting the applicants for the model to those with a particular occupation; and 
constructing the model as an occupationally-specialized model. 

6. (Original) The method of claim 1 wherein the model accepts one or more 
inputs, the method further comprising: 

identifying in the pre-hire information one or more characteristics that are 
ineffective predictors; and 

omitting the ineffective predictors as inputs to the model. 

7. (Original) The method of claim 1 wherein the pre-hire information 
comprises one or more characteristics, the method further comprising: 

identifying in the pre-hire information one or more characteristics that are 
ineffective predictors; and 

providing an indication that the characteristics no longer need to be collected. 

8. (Previously Presented) The method of claim 1 wherein job performance 
criteria predictions comprise a prediction indicating whether a job candidate will be 
voluntarily terminated. 
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9. (Previously Presented) The method of claim 1 wherein job performance 
criteria predictions comprise a prediction indicating whether a job candidate will be 
eligible for rehire after termination. 

1 0. (Previously Presented) The method of claim 1 wherein the pre-hire 
information comprises one or more characteristics, the method further comprising: 

identifying in the pre-hire information one or more characteristics that are 
ineffective predictors; 

responsive to identifying the ineffective predictors, collecting new pre-hire 
information not including the ineffective predictors; and 

building a refined model based on the new pre-hire information. 

1 1 . (Original) The method of claim 10 further comprising: 

adding one or more new characteristics to be collected when collecting the new 
pre-hire information. 

12. (Original) The method of claim 1 1 further comprising: 
evaluating the effectiveness of the new characteristics. 

13. (Canceled) 
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14. (Canceled) 

15. (Previously Presented) A method for constructing an artificial intelligence- 
based employment selection process based on pre-hire information comprising personal 
employee characteristics and post-hire information comprising employee job 
performance observation information, the method comprising: 

via information-theoretic feature selection, choosing questions from the pre-hire 
information as chosen features for which respective pre-hire applicant responses serve as 
inputs to predictive artificial models, wherein the information-theoretic feature selection 
comprises identifying at least one higher-order interaction comprising a set of a plurality 
of questions having higher predictive power than a sum of predictive powers of 
individual questions in the set; 

generating a plurality of predictive artificial intelligence models accepting the 
chosen features as inputs, based on the pre-hire and post-hire information, wherein at 
least two of the artificial intelligence models are of different feed-forward types and 
different architectures; 

testing effectiveness of the models to select an effective model; and 

applying the effective model to predict post-hire information not yet observed, 
whereby the post-hire information not yet observed that is predicted by the effective 
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model can be a basis for a hiring recommendation or other employee selection 
information. 

16. (Canceled) 

17. (Previously Presented) The method of claim 1 5 wherein at least one of the 
models is an expert system. 

18. (Canceled) 

19. (Canceled) 

20. (Canceled) 

21 . (Original) The method of claim 15 further comprising: 
identifying at least one of the models as exhibiting impermissible bias; and 
avoiding use of the models exhibiting impermissible bias. 

22. (Canceled) 
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23 . (Previously Presented) A computer-implemented method of refining an 
artificial-intelligence based employee performance selection system, the method 
comprising: 

collecting information via an electronic device presenting a set of questions to 
employment candidates, wherein the questions are stored in a computer-readable 
medium; 

collecting post-hire information for the employment candidates based on job 
performance of the employment candidates after hire, wherein collecting post-hire 
information comprises receiving payroll information for the employment candidates via a 
network, determining a termination date for an employment candidate from the payroll 
information received via the network, and determining tenure of the employment 
candidate by comparing the termination date with a hiring date of the applicant; 

testing effectiveness of at least one of the questions in predicting tenure, wherein 
testing effectiveness comprises considering various sets of permutations of predictive 
items comprising the at least one of the questions, wherein the at least one of the 
questions alone is ineffective but is effective in combination with one or more other 
items; and 

responsive to determining the question is ineffective in predicting tenure, deleting 
the question from the computer-readable medium. 

24. (Canceled) 
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25. (Previously Presented) A computer-readable medium comprising a 
predictive model, the model comprising: 

inputs for accepting one or more characteristics based on pre-hire information for 
a job applicant, wherein at least one of the inputs is chosen via information-theoretic 
feature selection and corresponds to a question for which pre-hire applicant responses 
have been collected, wherein the information -theoretic feature selection comprises 
identifying at least one higher-order interaction comprising a set of a plurality of 
questions having higher predictive power than a sum of predictive powers of individual 
questions in the set; and 

one or more predictive outputs indicating one or more predicted job effectiveness 
criteria based on the inputs, 

wherein the predictive model is an artificial intelligence-based model constructed 
from pre-hire data electronically collected from a plurality of employees and post-hire 
data, and the model generates its predictive outputs based on the similarity of the inputs 
to pre-hire data collected for the plurality of employees and their respective post-hire 
data; and 

wherein the one or more predicted job effectiveness criteria comprise at least one 
selected from the group consisting of; 

whether the job applicant will be involuntarily terminated, and whether the job 
applicant will be eligible for rehire upon termination. 
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26. (Original) The computer-readable medium of claim 25 wherein the 
predictive model comprises a predictive output indicating a rank for the job applicant, 

27. (Original) The computer-readable medium of claim 26 wherein the rank is 
relative to other applicants. 

28. (Original) The computer-readable medium of claim 26 wherein the rank is 
relative to the plurality of employees. 

29. (Original) The computer-readable medium of claim 25 wherein the 
predictive model comprises a predictive output indicating probability of group 
membership for the job applicant. 

30. (Original) The computer-readable medium of claim 25 wherein the 
predictive model comprises a predictive output indicating predicted tenure for the job 
applicant. 

31. (Canceled) 
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32. (Original) The computer-readable medium of claim 25 wherein the 
predictive model comprises a predictive output indicating predicted number of accidents 
for the job applicant. 

33. (Original) The computer-readable medium of claim 25 wherein the 
predictive model comprises a predictive output indicating whether the applicant will be 
involuntarily terminated. 

34. (Original) The computer-readable medium of claim 25 wherein the 
predictive model comprises a predictive output indicating whether the applicant will be 
eligible for rehire after termination. 

35. (Canceled) 

36. (Canceled) 
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37. (Previously Presented) A computer-readable medium comprising a refined 
predictive artificial intelligence-based model, the model comprising: 

inputs for accepting one or more characteristics based on pre-hire information for 
a job applicant, wherein at least one of the inputs is chosen via information-theoretic 
feature selection and corresponds to a question for which pre-hire applicant responses 
have been collected, wherein the information-theoretic feature selection comprises 
identifying at least one higher-order interaction comprising a set of a plurality of 
questions having higher predictive power than a sum of predictive powers of individual 
questions in the set; and 

one or more predictive outputs indicating one or more predicted job effectiveness 
criteria based on the inputs; 

wherein the one or more predictive outputs are configured to predict whether the 
job applicant will be involuntarily terminated or whether the job applicant will be eligible 
for rehire upon termination, and 

wherein the predictive model is constructed from pre-hire data electronically 
collected from a plurality of employees and post-hire data, wherein the pre-hire data is 
based on a question set refined by having identified and removed one or more questions 
as ineffective. 



38. (Canceled) 
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39. (Canceled) 

40. (Previously Presented) An apparatus for assisting in determining the 
suitability of an individual for employment by an employer, the apparatus comprising: 

a computer-readable medium comprising a predictive model comprising: 
inputs for accepting one or more characteristics based on pre-hire information for 
a job applicant, wherein at least one of the inputs is chosen via information-theoretic 
feature selection and corresponds to a question for which pre-hire applicant responses 
have been collected, wherein the information-theoretic feature selection comprises 
identifying at least one higher-order interaction comprising a set of a plurality of 
questions having higher predictive power than a sum of predictive powers of individual 
questions in the set; and 

one or more predictive outputs indicating one or more predicted job effectiveness 
criteria based on the inputs, wherein the predictive model is an artificial intelligence- 
based model constructed from pre-hire data electronically collected from a plurality of 
employees and post-hire data, and the model generates its predictive outputs based on the 
similarity of the inputs to pre-hire data collected for the plurality of employees and their 
respective post-hire data; wherein the one or more predicted job effectiveness criteria 
comprise at least one selected from the group consisting of: whether the job applicant will 
be involuntarily terminated, and whether the job applicant will be eligible for rehire upon 
termination; 
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electronic data interrogator means for presenting a first set of a plurality of means 
for questioning to the individual; 

electronic answer capturer means for electronically storing answers by the 
individual to at least a selected plurality of the first set of means for questioning 
presented to the individual; 

electronic predictor means responsive to the stored answers and for predicting at 
least one post-hire outcome if the individual were to be employed by the employer, the 
predictor providing a prediction of the outcome based upon correlations of the stored 
answers with answers to sets of means for questioning by other individuals for which 
post-hire information has been collected; 

electronic results provider means for providing an output indicative of the 
outcome to assist in determining the suitability of the individual for employment by the 
employer. 

41. (Previously Presented) An apparatus for assisting in determining the 
suitability of an individual for employment by an employer, the apparatus comprising: 

an electronic device for presenting a first set of a plurality of means for 
questioning to the individual; 

means for electronically storing answers by the individual to at least a selected 
plurality of the first set of means for questioning presented to the individual; 
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means responsive to the stored answers and for predicting at least one post-hire 
outcome if the individual were to be employed by the employer, the predictor providing a 
prediction of the outcome based upon correlations of the stored answers with answers to 
sets of means for questioning by other individuals for which post-hire information has 
been collected; and 

means for providing an output indicative of the outcome to assist in determining 
the suitability of the individual for employment by the employer, 

wherein the apparatus comprises a predictive model comprising: 

inputs for accepting one or more characteristics based on pre-hire information for 
a job applicant, wherein at least one of the inputs is chosen via information-theoretic 
feature selection and corresponds to a question for which pre-hire applicant responses 
have been collected, wherein the in formation -theoretic feature selection comprises 
identifying at least one higher-order interaction comprising a set of a plurality of 
questions having higher predictive power than a sum of predictive powers of individual 
questions in the set; and 

one or more predictive outputs indicating one or more predicted job effectiveness 
criteria based on the inputs, 
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wherein the predictive model is an artificial intelligence-based model constructed 
from pre-hire data electronically collected from a plurality of employees and post-hire 
data, and the model generates its predictive outputs based on the similarity of the inputs 
to pre-hire data collected for the plurality of employees and their respective post-hire 
data; and 

wherein the one or more predicted job effectiveness criteria comprise at least one 
selected from the group consisting of: 

whether the job applicant will be involuntarily terminated, and whether the job 
applicant will be eligible for rehire upon termination. 

42. (Canceled) 

43. (Previously Presented) The computer-readable medium of claim 37 
wherein the ineffective questions are identified via an information transfer technique 
comprising: 

considering various sets of permutations of answers to questions; 
evaluating whether the sets of permutations are effective; and 
determining whether an answer to a question is not in any effective set of 
permutations. 
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44. (Previously Presented) The method of claim 46 wherein collecting post-hire 
information further comprises: 

tracking whether the applicant has been dropped from payroll , 

45. (Previously Presented) The method of claim 21 wherein: 

the identifying comprises comparing across gender, ethnicity, or age differences. 

46. (Previously Presented) The method of claim 1 wherein: 

collecting post-hire information comprises receiving payroll information for the 
applicants via a network, determining a termination date for an applicant from the payroll 
information received via the network, and determining tenure of the applicant by 
comparing the termination date with a hiring date of the applicant, 

47. (Previously Presented) The method of claim 1 wherein: 

the set of the plurality of questions having higher predictive power than the sum of 
predictive powers of individual questions in the set comprises at least one biodata 
question and at least one psychometric question. 
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48. (Previously Presented) The method of claim 1 wherein identifying at least 
one higher-order interaction comprises: 

generating an approximation of an optimal subset of quetsions for use as input 
features for the model. 

49. (Previously Presented) The method of claim 48 wherein generating an 
approximation of an optimal subset of questions comprises: 

determining a union of a set of questions appearing in predictive transmissions of 
greatest magnitude. 

50. (Previously Presented) The method of claim 48 wherein generating an 
approximation of an optimal subset of questions comprises: 

from a set of transmissions T k , choosing m unique transmissions of greatest 
magnitude as a base set for higher-order transmissions; 

generating T\ +1) by adding questions to members of T^, that generate a set T'k+i 
with largest transmission values; and 

taking a union of questions appearing in the unique transmissions of greatest 
magnitude, wherein the union approximates the optimal subset of questions. 
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51. (New) One or more computer-readable storage media having stored 
thereon an executable model operable to generate one or more job performance criteria 
predictions based on input pre-hire information, the model constructed via a method 
comprising: 

electronically collecting pre-hire information from a plurality of applicants 
wherein at least some of the pre-hire information is collected from at least one of the 
applicants who responds directly on an electronic device to provide pre-hire applicant 
responses to questions; 

collecting post-hire information for the applicants based on job performance of the 
applicants after hire; 

via information-theoretic feature selection, choosing questions from the pre-hire 
information as features for which respective pre-hire applicant responses serve as inputs 
to the model, wherein the information-theoretic feature selection comprises identifying at 
least one higher-order interaction comprising a set of a plurality of questions having 
higher predictive power than a sum of predictive powers of individual questions in the 
set, wherein the higher-order interaction exhibits a synergy between the set of the 
plurality of questions having higher predictive power; 

from the pre-hire information and the post-hire information, training an artificial 
intelligence-based predictive model in a computer-readable medium with observed pre- 
hire applicant responses for the chosen features, wherein the artificial intelligence-based 
predictive model is operable to generate one or more job performance criteria predictions 
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based at least on input pre-hire information from new applicants corresponding to the 
chosen features, whereby the one or more job performance criteria predictions are usable 
as a basis for a hiring recommendation or other employee selection information; 

deploying the model, wherein deploying comprises converting the model into 
command code and providing an operational applicant processing system; and 

conducting performance tuning for the model, wherein performance tuning 
comprises continuing data collection, monitoring sample size as incoming data 
accumulates, and repeating feature selection. 
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